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Abstract. Bruinier and Yang conjectured a formula for an intersection number on the 
arithmetic Hilbert modular surface, CA4{K).Tm, where CAi{K) is the zero-cycle of points 
corresponding to abelian surfaces with CM by a primitive quartic CM field K, and Tm is 
the Hirzebruch-Zagier divisors parameterizing products of elliptic curves with an m-isogeny 
Ch . between them. In this paper, we examine fields not covered by Yang's proof of the conjec- 

ture. We give numerical evidence to support the conjecture and point to some interesting 
anomalies. We compare the conjecture to both the denominators of Igusa class polynomials 
and the number of solutions to the embedding problem stated by Goren and Lauter. 
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! 1. Introduction 

Hilbert class polynomials are polynomials whose zeros are j-invariants of elliptic curves 
with complex multiplication (CM). Similarly, Igusa class polynomials are polynomials whose 
zeros are Igusa invariants of genus 2 curves whose Jacobians have CM by a quartic CM 
field K. Igusa class polynomials can be hard to compute, mostly because recovering the 
(yQ • coefficients from approximations requires a bound on the denominators. A precise description 
O . of the denominators would greatly improve the estimated running time of current algorithms 
^ ! to compute Igusa class polynomials |Spa94il7W99llWen03l[ELllGHK+06 . 
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The first attempt to understand the primes appearing in the factorization of the denom- 
■ inators was given in [Lauj . where it was conjectured that the primes are bounded by the 
O . absolute value of the discriminant dx of the field K, and that the primes divide dx — x"^, for 
some integer x. The divisibility condition actually follows from work of Goren |Gor97] along 
with the observation that primes in the denominators are primes of superspecial reduction 
^ ' for the abelian surfaces with CM by K. In |GL07] . Goren and Lauter proved a bound on 
the primes which appear in the factorization of the denominators. In recent work |GL] . they 
have given a bound on the denominators (i.e. bounding the powers of the primes which 
appear); however, this bound is not sharp. 

Igusa invariants can be expressed as rational functions in Eisenstein series and cusp forms. 
Using this description it can be shown that the primes dividing the denominators are (up 
to cancellation) the primes p such that there exists a genus 2 curve C with CM by K and a 
prime ideal p\p with 

Jac(C)^ExE' (modp). 

Goren and Lauter obtained the bound on such primes by showing that these primes are 
exactly the primes for which there is a solution to the embedding problem: the problem 
of giving an embedding Ok End(-E' x E') C M2(Bp^oo), where E, E' are supersingular 
elliptic curves over Fp and Bp^oo is the quaternion algebra over Q ramified at only p and oo. 
Such decomposable Jacobians can be viewed as points on the Siegel moduli space where the 
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arithmetic intersection of the CM(ii')-cycle with the divisor of the cusp form xio is nonzero 
at p. 

This arithmetic intersection number on the Siegel moduh space can be related to a sum of 
intersection numbers of Hirzebruch-Zagier divisors and CM(i^)-cycles on the Hilbert mod- 
ular surface associated to the real quadratic subfield of K. In |BY06j . Bruinier and Yang 
conjectured a formula for the intersection number CAi{K).Tm, where CM.{K) parameterizes 
abelian surfaces with complex multiplication by a primitive quartic CM field and 7^ is a 
Hirzebruch-Zagier divisor parameterizing products of elliptic curves with an m-isogeny be- 
tween them when m is split in the real quadratic field. The results of Bruinier and Yang can 
be viewed as a generalization of the celebrated formula of Gross and Zagier |GZ85] . Bru- 
inier and Yang conjectured their formula for these arithmetic intersection numbers under 
the assumption that the real quadratic subfield of K has prime discriminant. Their formula 
relates the intersection numbers to the number of ideals of a certain norm in the reflex field 
of K. In lYanatlYanb] , Yang gives a detailed treatment of the embedding problem and uses 
it, along with other techniques, to prove the conjectured intersection formula under certain 
conditions on the CM field K. 

For a given CM field K, the conjectural formula of Bruinier and Yang is straightforward 
to compute. Thus, if the Bruinier- Yang formula held for all the formula would give a 
way to compute the denominators precisely (up to cancellation). 

In this paper, we compare the number of solutions to the embedding problem, the denom- 
inators in the Igusa class polynomials, and the conjectural formula of Bruinier and Yang for 
the 13 CM fields found in [vW99j . Only 6 of these fields are covered by Yang's proof of the 
conjecture. We give numerical evidence to support the conjecture and point to some inter- 
esting anomalies and some possible explanations for these discrepancies. Such a project was 
suggested in the Introduction and Section 9 of |BY06j . and we give here the first numerical 
evidence comparing the conjecture with denominators of Igusa class polynomials and count- 
ing solutions to the embedding problem. This comparison required developing an algorithm 
to count the number of solutions to the embedding problem for a given primitive quartic 
CM field K and prime number p. The algorithm is given in Section 16.11 and the Magma code 
for computing the values is given in the Appendix. 

The paper is organized as follows. Section |5] contains background on abelian surfaces with 
complex multiplication by a primitive quartic CM field K. Section [3] contains a description 
of Igusa invariants and Igusa class polynomials. It also gives a brief introduction to the 
algorithms for computing Igusa class polynomials and their connection with generating genus 
2 hyperelliptic curves for cryptography. Section H] introduces arithmetic intersection theory 
on the arithmetic Hilbert moduli space and explains how it relates to the denominators of 
Igusa class polynomials. In this section, we also describe the Bruinier- Yang conjecture and 
the cases in which it is known. Section |5] explains how the denominators of Igusa class 
polynomials relate to the embedding problem. Section E] gives an algorithm to count the 
number of solutions to the embedding problem. Section [7] presents our numerical data and 
analysis of the findings. 

This paper is concerned with computing these intersection numbers from three different 
points of view: denominators of Igusa class polynomials, arithmetic intersection numbers via 
the conjectured Bruinier- Yang formula, and embeddings of a CM number field into a matrix 
algebra over a quaternion algebra. Thus the paper necessarily requires the introduction of 
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a number of objects from different points of view, and the background is spread over many 
different references and papers. Therefore, for the readers' convenience, we have tried to 
include here as much background as possible. Most sections are self-contained, but Section |4] 
leaves some of the background material to the references. 

2. Abelian surfaces with cm by Ok 

A CM field is a totally imaginary quadratic extension of a totally real number field of finite 
degree. For a CM field K, let denote the real quadratic subfield of K. If . . . , 02n} are 
the 2n complex embeddings of K, then a CM-type $ for is a choice of n distinct complex 
embeddings, no two of which are complex conjugates of one another. When = Q, CM 
fields are imaginary quadratic fields. 

An abelian variety with CM by K is a. pair (A, l) where A is an abelian variety and l is 
an isomorphism of K into End°(yl) := End(/1) (g) Q. We say {A,l) has type (K,^) if A 
has CM by K and the analytic representation of End°(yl) is equivalent to the direct sum of 
the n isomorphisms </)j G $. One can define a CM-type to be primitive if all of the abelian 
varieties over C of that type are simple, i.e. have no abelian subvarieties other than {0} 
and itself. A CM-type {K, $) is primitive if and only ii K = K^{^), where ^ is an element 
of K such that is totally negative and Im(^''^') > for i = 1, . . . , n |Shi98l p. 61]. For 
quartic CM fields, the only case where non-primitive CM-types arise is the case when K is 
biquadratic |Shi98t p. 64]. We will restrict ourselves to the primitive (i.e. non-biquadratic) 
case in this article. 

Let {K, $) denote a quartic CM field K with CM-type $. Define the reflex field of (K, $) 
to be 

If K is Galois cyclic, then we have K = lU f fShiQSl §8.4]). 

Shimura gives a construction of a principally polarized complex abelian variety of a given 
CM type in |Shi98t §6.1, Theorem 2]: to any ideal I C Ok-, associate the abelian variety 
Ai = C^/$(/) of dimension 2. It has endomorphism ring Ok and we say it has CM by 
Ok- All abelian varieties with CM by Ok are obtained via this construction. A principal 
polarization of A[ is an isomorphism between Aj and its dual Aj = /^{I T>~^), where 

V-^^ = {xeK\ Tik/q{xOk) C Z} 

is the inverse different. If vr is a totally imaginary element of K which satisfies $(7r) G (iM>o)^ 
and Till = T)~^, then the map Aj — Ai given by 

(21,22) H- (0i(vr)2i,02(vr)z2) 

is an isomorphism ( |Shi98t p. 102-104]). 

Let Khe a. fixed primitive quartic CM-field and fix a CM-type $ : — C^. Let A/<C be a 
principally polarized abelian surface of CM-type {K, $) with endomorphism ring isomorphic 
to Ok- We define a group ^{K) = {{a, a)}/ ~ where a is a fractional ideal of Ok such that 
ad = (a), with a G totally positive. Two pairs (a, a) and (b,/3) are equivalent if and 
only if there exists a unit u G K* with b = ua and (3 = una. The multiplication is defined 
componentwise, and {Ok, 1) is the identity element of '^{K). 
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The group €{K) naturally acts on the finite set S{K, $) of isomorphism classes of princi- 
pally polarized abelian surfaces that have CM by Ok of a given type $. Every principally 
polarized abelian variety with CM by K has an analytic representation Aj determined by 
an ideal / and a ^-positive element tt & K giving the principal polarization. We now put 

(a, a) ■ (/, tt) = (a/, an) 

for (a, a) E ^{K). By |Shi98l §14.6], for a given $ the action of on S{K, $) is transitive 
and free. In particular, we have = |>S'(A', $)|, and the sets are often identified. 

Abelian surfaces A of type {K, $) defined by an ideal a have potentially zero, one, or 
two principal polarizations. There are zero principal polarizations only if there is no totally 
positive element a G Okq with aa = (a). Otherwise there are [U : Ui] principal polarizations 
on A, where U is the group of all totally positive units of Kq, and Ui is the subgroup of 
norms of units from K. 

For quartic primitive CM fields K, there are two different possible CM types up to complex 
conjugation. Let p be complex conjugation, and let be a non-trivial embedding of K into 
C other than complex conjugation. A priori, the 4 possible CM types are: $ = (1,0), 
$ = {p,p4>), $' = (1,0), $ = {p,P<P)- We denote by K<s, (resp. K^pr, etc.) the set of 
principally polarized abelian varieties of CM type $ (resp. etc.). 

The description above identifies isomorphism classes of principally polarized abelian vari- 
eties of a given type $ with the set {(a, a)}/ ~, where o is an ideal of Ok such that aa = (a), 
a a totally positive element of Okq- The associated abelian variety is A = C^/<l>(a) and the 
embedding of Ok into End(y4) sends /3 G Ok to the diagonal matrix with {(3'^^ , P"^^) on the 
diagonal acting by multiplication on C^. 

We can show that = K-^ by sending (a, a) to (a, a). For the same reason, Kq,' = K-^i. 
If K is cyclic, then 0^ is complex conjugation and sending (a, a) to (a''^, a'^) shows that 
K<^ = K^i. 

3. Genus 2 curves with CM 

In cryptography, it is useful to be able to construct genus 2 curves over a finite field 
Fg with a given number of points on its Jacobian, since they can be used to implement 
discrete-log cryptosystems. Many algorithms to generate such curves have been developed 
using complex multiplication, all of which require computing certain invariants first studied 
by Igusa |Igu60| . The closely related Igusa class polynomials have rational coefficients. The 
known algorithms for computing Igusa class polynomials all require a good bound on the 
primes appearing in the denominators of these invariants. One of the main goals of this 
project is to better understand exactly which primes appear in those denominators. In this 
section, we will review the definitions of the Igusa invariants and explain several algorithms 
for computing Igusa class polynomials. 

3.1. Generating genus 2 curves for cryptography. To construct a genus 2 curve C over 
a finite field Fg with a given number of points on its Jacobian, one method is to compute 
a related quartic CM field K such that a curve C with #J(C)(Fg) = A^ has CM by K. 

For an ordinary genus 2 curve C over a prime field F,, let A"! = ^C{¥g) and N2 = #C(Fg2). 
Then 



(1) 



i^J{C)(¥,) = {N^ + N,)/2-q. 
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To find a curve C over Fg such that ^J{C) = N, first find A'"! and N2 in the Hasse-Weil 
intervals for Fg and ¥g2 satisfying relation ([1]). Next, set 

si := g + 1 - iVi 

and 

S2 := ^ (s? + iV2 - 1 - q') ■ 

Then the quartic polynomial f{t) = — Sit^ + ^2^^ — qsit + is the Weil polynomial of a 
genus 2 curve as long as the exceptional cases listed in |HNR09l Theorem 1.2] are avoided. 
Note that if S2 is prime to q, then the Jacobian is ordinary |How95t p. 2366]. Under those 
conditions and assuming that f{t) is an irreducible polynomial in Q[t], the Jacobian of the 
curve has endomorphism ring equal to an order in the quartic CM field K = Q[t]/{f{t)). 

Also, if K can be written in the form K = Q(z\/ a + by/d), with a,b,dE7j and d and (a, b) 
square-free, then is a primitive CM field if and only if a^—b'^d is not a square |KW89t p. 135]. 

To find a curve with CM by K, one method is to compute its Igusa invariants by evaluating 
certain Siegel modular functions at CM points associated to K in the Siegel moduli space. 
Igusa invariants will be explained in the next section. 



3.2. Igusa invariants and Igusa class polynomials. Recall that the j-invariant of an 
elliptic curve can be calculated in two ways. One can compute it as a value of the modular 
j-function on a lattice defining the elliptic curve as a complex torus over C, or directly 
from the coefficients of the equation defining the elliptic curve. Similarly, the three Igusa 
invariants of a genus 2 curve can be calculated in two different ways. 

Let = f{x) be a hyperelliptic curve, where f{x) is a sextic with roots ai, . . . , ae and 
leading coefficient a^. Igusa defined invariants A,B,C,D as certain symmetric functions of 
the roots |Igu60| . For example, D = al^ Y[i<ji(^i ~ Q^i)^ is the discriminant. The ring of 



rational functions of the coarse moduli space for hyperelliptic curves of genus 2 is generated 
by the absolute Igusa invariants, which can be defined as: 

A^B A^C 

(2) H:=^, ^-=^' ''-■=^- 

This choice of generators is not unique. These invariants are a generalization of the j- 
invariant for elliptic curves in the sense that, if ii is non-zero and the characteristic is not 
2 or 3, these invariants agree for two smooth genus 2 curves if and only if the two curves 
are isomorphic over an algebraically closed field. Note that if /(x) defines a smooth genus 2 
curve, then 21,22, "^3 are well defined. 

The invariants can also be defined in terms of modular functions on the Siegel moduli 



space. In Igu62 p. 195], Igusa defined normalized Siegel modular cusp forms Xio and X12 
of weights 10 and 12 in terms of the Siegel-Eisenstein series for w = 4,6, 10, 12. There 
he used the term normalized to mean that the leading coefficient of the Fourier expansion 
is 1. For our purposes, we will want to work with modular forms that have integral Fourier 
coefficients which are relatively prime. By working with 4xio and 12xi2, we obtain modular 
forms normalized in this second sense. 
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In |GL07l Section 5.2], it was noted that the absolute Igusa invariants defined above, ii, 
22, "^3, can also be expressed in terms of these Eisenstein series and cusp forms as follows: 

O q5Xi2 ■ cy~3 q3-^4Xi2 ■ cy-5 o f ^6X12 . r,2 0-^4X12 

tl — ^ ■ o — g-, l2 — ■i' ■ o — I — , %z — I ■ o I — 3 V I ■ o — I — 

Xio Xio V Xio Xio 

(where the parentheses in the expression for Z3 were mistakenly left out in |GL07j ). It was 



also remarked there that it follows from the formulas given by Igusa in Igu67 p. 848] that 
these ii, i2, "^s coincide with the invariants defined in terms of theta functions and used for 
computation by |vW99tlWen03t[Lau] . However, it is worth noting that in order to work with 
invariants which are quotients of normalized modular forms in the second sense, one should 
actually work with 2~^ii, 2z2, 2^i3 as Spallek did |Spa94| . 

Igusa class polynomials are the genus 2 analogue of Hilbert class polynomials. Just as we 
need three Igusa invariants to define an isomorphism class instead of one j-invariant, we have 
a triple of Igusa class polynomials for a CM field K instead of one Hilbert class polynomial. 
This triple of polynomials is defined as follows. 

(3) He{X):=l[{X-z,{T)), i = 1,2,3, 

T 

where the ii are the absolute Igusa invariants defined above, and the product is taken over 
all r G Sp(4, Z)\f)2 such that the associated principally polarized abelian variety has CM 
by Ok- Igusa class polynomials have rational coefficients |Spa94[ Satz 5.8] (as opposed to 



integral coefficients as in the case of Hilbert class polynomials). In the next section, we 
describe how this plays a key role in the difficulty of computing these polynomials. 

3.3. Algorithms for computing Igusa class polynomials. The complex analytic method 
for computing genus 2 curves with a given number of points on the Jacobian is analogous 
to the Atkin-Morain algorithm for elliptic curves |AM93j . Recall that the Atkin-Morain 
algorithm computes the Hilbert class polynomial of an imaginary quadratic field K by eval- 
uating the modular j-invariants of all elliptic curves with CM by K to high precision. The 
complex analytic method |Spa94[lvW99ilWen03j for generating genus 2 curves takes as input 



a primitive quartic CM field and evaluates the Igusa invariants to high precision to form the 
class polynomials with coefficients in Q. Computing the Igusa invariants to high precision 
is difficult to do efficiently. To give an algorithm requires a bound on the denominators of 
the class polynomials, and no such bound was known at the time of those papers. 

Given a triple of roots of these polynomials, Mestre gave an algorithm to recover the 
equation of a genus 2 curve from its invariants |Mes91j . So to find a genus 2 curve, first 
find roots of the polynomials modulo a prime number p and then use Mestre's algorithm to 
construct a genus 2 curve over Fp whose Jacobian has CM by K. 

In |EL] . Lauter and Eisentrager present a CRT algorithm, an alternative to the complex 
analytic algorithm to compute the Igusa class polynomials. The CRT algorithm takes as 
input a quartic CM field K and a constant c such that all denominators divide c. The 
algorithm first computes the Igusa class polynomials modulo small primes i by finding all 
curves with the right zeta function and the right endomorphism ring modulo i. It then 
forms the three polynomials whose roots are the invariants of those curves. Finally, it 
uses the Chinese Remainder Theorem (CRT) and the constant c to obtain the Igusa class 
polynomials with coefficients in Q. 
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There is another alternative approach to computing Igusa class polynomials using p-adic 
arithmetic ( [GHK"'"06] ). The method is similar to the complex analytic algorithm, except 
the values are calculated to a high p-adic precision, instead of a complex precision. This 
algorithm also requires a bound on the denominators. 

Notice that all three algorithms require a bound on the denominators, and their running 
time is directly affected by the sharpness of the bound. In the complex analytic and p-adic 
algorithms, it is required to bound the amount of precision needed in the approximation 
of the jj-values. In the CRT algorithm, a common multiple of the denominators is needed. 
Thus, an exact formula for the denominators would be very useful. 

4. Arithmetic intersection theory and denominators of Igusa class 

polynomials 

In this section we describe how the denominators of Igusa class polynomials are related to 
arithmetic intersection numbers on the Siegel moduli space of principally polarized abelian 
surfaces and explain how this relates to intersection numbers on the Hilbert modular surface 
and the conjectural formula of Bruinier and Yang for the intersection number CA4{K).Tm- 

It is a classical fact that the zero locus of Xio on the coarse moduli space of abelian 
surfaces consists of exactly those abelian surfaces that decompose as a product of elliptic 
curves with the product polarization. The arithmetic analogue of this statement was proved 
in |GL07t Cor 5.1.2], that if a prime p divides the denominator of (//(4xio)'^)(r), for r a 
CM point corresponding to a smooth curve C and / a Siegel modular form of weight 10k 
with integral Fourier coefficients with gcd 1, then C has bad reduction modulo p. 

The absolute Igusa invariants are defined above as rational functions of the Siegel modular 
forms XiO; Xu, ^4, and ipQ, with only powers of Xio in the denominators. Although this 
definition of the invariants is given in terms of genus 2 curves over C, the Igusa invariants 
can also be defined for genus 2 curves over finite fields (as in formula [2] above). Furthermore, 
the Igusa invariants evaluated at CM points are algebraic numbers and can be reduced 
modulo suitable primes. Avoiding bad primes in the denominators, we have [ELI Thm 2] 

ie{C mod p) = ie{C) mod p. 

Thus the valuation of the denominator of the constant term of Hi at a prime p is the sum of 
the order of the poles of 2 ■ at the CM-points over F„. We have an analogous statement 

for the Igusa invariants H2 and H3. Since xu, ipi^ and ipQ have no poles, the orders of the poles 
of ii are bounded above by suitable multiples of the order of zeros of Xio- Experimentally, 
we observe that usually Xi2,4'4, and ipQ do not share zeros of high multiplicity with xio, 
so the order of the poles should be close to the order of zeros of a suitable power of xio- 
Computing the order of zeros of xio is equivalent to computing the arithmetic intersection 
number dw{xio) -CMIK) (explained below) of the divisor of xio with the cycle of CM points 
associated to K. 

To compute the arithmetic intersection number diY{xio).CAi{K), first note that, up to 
a power of 2, xio = ^i,siegei ( |BY06t Remark 9.3]), where \l/i,siegei is the normalized Siegel- 
Borcherds product of discriminant 1 whose divisor is Gi, the Humbert surface of discriminant 
1. Thus at all primes except possibly 2, we need to compute 2Gi.CA4{K). The Humbert 
surface Gi can be pulled back to a sum of Hirzebruch-Zagier divisors on the Hilbert 
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moduli space, and so the quantity to be computed can be expressed as a sum of intersection 
numbers of the form CA4{K).Tm on the Hilbert moduh space. 

We now explain all of this in more detail. The presentation in the rest of this section follows 
closely the treatment in [YanallYanbtlBBCKOT] . For further background, good references are 
the books of |vdG88t[Gor02tlK"RY06j . In Section Wl] we describe the Hilbert moduli space 
and the moduli interpretation of Hirzebruch-Zagier divisors and CM-cycles. In Section \4.2\ 
we explain arithmetic intersection theory and the meaning of CM.{K).%n- In Section [4.3^ 
we explain the relationship between the Humbert surface and Hirzebruch-Zagier divisors. In 
Section we describe the Bruinier-Yang conjectural formula for these intersection numbers. 

4.1. The Hilbert moduli space and Hirzebruch-Zagier divisors. Fix F, a real qua- 
dratic extension of Q with different Vp. The Hilbert moduli space parametrizes principally 
polarized abelian surfaces A with real multiplication by Op, i.e. with a fixed embedding 
l: Of End(A). More precisely, the arithmetic Hilbert moduli space Ai is the moduli 
stack that associates a scheme S over Z to the set of triples {A, t, A) where 

(1) A is an abelian surface over S, 

(2) l: Op ^ Ends{A) gives the structure of real multiplication, and 

is a homomorphism of OiT'-modules that takes totally 
positive elements to Oi^-linear symmetric polarizations and satisfies the Deligne- 
Pappas condition that Vp^ (g) A — is an isomorphism of abelian schemes. 

The C-points of Ai can be identified with the usual Hilbert moduli space M = SL2{Of)\S}'^ 
described in the article of Grundman in this volume |Gru] . 

Now suppose that m is a prime number that splits in F. Let Yolm) = ro(m)\i3 be the 
modular curve parameterizing pairs of elliptic curves with an m-isogeny. Let yoim) be the 
moduli stack over Z[l/m] of pairs of elliptic curves with an m-isogeny. Bruinier, Burgos 
Gil, and Kiihn |BBGK07t Prop 5.11, 5.13] construct a map from yo{m) into Ai. They 
prove that the image of the minimal compactification of yo{m) is the Zariski closure of the 
Hirzebruch-Zagier divisor, Tm- 

Let i^' be a primitive quartic CM number field with = F. Let CA4.{K) be the moduli 
stack over Z that represents the moduli problem assigning a base scheme S to the set of triples 
[A, i, A), where l: Ok — ^ E^ds{A) gives the (9x-action on A such that the Rosati involution 
defined by A induces complex conjugation on Ok, and {A,l\op,X) G J^{S). Bruinier and 
Yang also refer to the direct image of CAi{K) in Ai by CAi{K). 

4.2. Arithmetic Intersection Theory. Following [KRY06[ Chapter 2] and [YanallYanbj . 

we define the arithmetic intersection number of two cycles on a Deligne-Mumford stack of 
dimension n that is proper and flat over Z. Let Zi and Z2 be two cycles that intersect 
properly, i.e. their codimensions are ri and r2, with ri + r2 = n, and Zid Z2 = Zi Xjn 2^2 is 
a Deligne-Mumford stack of dimension 0. Then the arithmetic intersection number (written 
logarithmically) is 




p xeZinZ2(¥p) 



1 



ip{Zi,Z2,x) logp. 



# Aut(x) 
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where OzinZ2,x is the local ring of Zi fl Z2 at x, and ip{Zi, Z2,x) = Length OzinZ2,x is the 
local intersection at x. 

We are interested in the arithmetic intersection of the cycles CM.{K) and T{m) on the 
moduli stack ^A.. Using the moduli interpretation of the cycles CM.{K) and Tm, it is 
clear that a geometric point x in CJ^{K) fl 7^ = CJ^{K) Tm is indexed by a pair 
{(f): E — E',l), where (j) G yQ{'m)(Wp) and t: Ok Endc)^(i?) is an embedding such that 
the Rosati involution associated to A induces complex conjugation on the image of K and 
where (5, A) is the polarized abelian surface in Tm which is the image of 0. Denote by 
ip{4>, i) = ip{CAi{K), T{m),x) the local intersection number at a point x = (0, l). It follows 
from deformation theory that ip{(j),L) = Length (W[[t, t']]//), where W is the Witt ring of 
¥p and / is the minimal ideal such that </> can be lifted to an isogeny modulo I and the 
embedding l can be lifted to an embedding modulo / |Yanb[ Eq. (3.7)]. 

4.3. The relationship between the Hilbert and Siegel moduli spaces. Let M be the 

Hilbert modular surface associated to the real quadratic field of discriminant D, F = Q{\/D). 
Recall that forgetting real multiplication gives a morphism (p^ from the Hilbert moduli space 
M to the Siegel moduli space X = Sp(4, Z)\f)2. The image Gd C X of M is known as the 
Humbert surface of discriminant D. For every Gd, there is a normalized integral Siegel 
Borcherds product '^D,siegei whose divisor is equal to Gd (up to multiplication by ±1). Let 
Qd denote the flat closure of in the Siegel moduli stack representing principally polarized 
abelian surfaces. 

The pullback to the Hilbert modular surface M of a Humbert surface Go' under the 
morphism (pD is a linear combination of Hirzebruch-Zagier divisors |vdG88t p. 215]: 

(4) (j)}j{GD') = ^ T^DD'-x2)/4, 

x<VDD' 
x2=DD'(4) 

for D' a positive discriminant, and the same is true of the fiat closures of these cycles in Ai. 
By Theorem 2.8 in |BY06] we also know that the pullback to M of the Borcherds product 
^D', Siegel is a Hilbcrt-Borcherds product with divisor (j4]). In our case, the denominators of 
the Igusa functions are powers of Xio? and Xio = ^i,sicgci ^ power of 2 and its divisor 

is 2Gi. So we can now express the quantity that we want to compute, 2Qi.CAi{K) as a sum 

2)/4. 

a;>0 

— 4 — e/.>o 

4.4. Bruinier-Yang Conjecture. Let K = F(a/A) be a non-biquadratic quartic CM field 
K, let $ be a CM type, and let {K, $) be a fixed reflex field and type. Bruinier and Yang 
assume that F = Q(-\/zJ) is a real quadratic field with D a prime congruent to 1 modulo 4. 
The real quadratic subfield oi K is F = Q{\/ AA'), where A' is the real conjugate of A. Let 
D be the norm of the relative discriminant of K/F and let dj^^p be the relative discriminant 

of K/F. Bruinier and Yang's conjectural formula for the arithmetic intersection number on 
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the Hilbert moduli space is: 



p 



where 



hm{p) logp = 




t= 



The ideals p\p are ideals of Op, and 




(ordpt + l)p(trf^^/^p-i) 



if p is split in K 
) log IpI if P is not split in K 



where p(a) = #{2t C | iV^/i?2l = a}. 

Yang has proved the conjecture [YanallYanbj under the assumptions that D = 1 mod 4 
is prime, A A' = 1 mod 4 is prime, and Ok is a free Oi^-module generated by 1 and 



norm of the relative discriminant of K/F under his assumptions. They are not necessar- 
ily equal in general, in particular, they differ for many of the fields listed in |vW99] . In 
our computations, we use the convention from |BY06j . that D = Np/qiDiscK/p) and we find 
that with this convention, the predicted arithmetic intersection number agrees with the com- 
puted values for the number of embeddings and the denominators of Igusa class polynomials 
in many cases. 



In this section, we explain the connection given in |GL07] between denominators of Igusa 
class polynomials and certain embeddings of the ring of integers of K into the matrix algebra 
M2 (Bp 00)7 where p is a prime number and Bp 00 is the unique (up to isomorphism) quaternion 
algebra over Q which is ramified at only p and 00. 

Let C be a smooth projective genus 2 curve such that Jac(C) has CM by Ok- There is 
a number field L such that all of the endomorphisms of Jac(C) are defined over L and thus 
Jac(C) has everywhere good reduction. It is observed in |GL07l §4.1] that if i^' is a primitive 
CM field and C has bad reduction at a rational prime p, then for a prime p\p of L, the 
reduction of Jac(C) modulo p is isomorphic over Ol/P to a product of supersingular elliptic 
curves Ei x E2 with the product polarization. Moreover, we have that Ei is isogenous to E2 
as otherwise 



contradicting K End°(Jac(C)) with K primitive. Goren and Lauter prove that in fact the 
existence of a smooth projective genus 2 curve C that has CM by Ok and has bad reduction 
at p is equivalent to the existence of an embedding 




some w G Op. Note that Yang sets D = AA', which is equal to the 



5. Quaternion algebras and embeddings of CM fields 



End(^i X E2) = End(Ei) x End(^2) 



l:Ok^ End(Ei x E2) 



such that the Rosati involution coming from the product polarization induces complex con- 
jugation on Ok |GL07l Lemma 4. LI and Thm. 4.2.1]. 

10 



5.1. Relating End(-E'i x E2) to a subring of M2(Bp_oo)- We view an element / E End(-E'i x 

E2) as a matrix (^j'^ /^^) where fij G B.om{Ej, Ei). Then the composition of endomor- 

phisms fog corresponds to multiplication of the matrices and the Rosati involution f ^ 
corresponds to 

f fl,l fl,2\ ^ ( fl,l f2,l\ 
V/2,1 /2,2y \fl,2 12,2/ 

Fix a supersingular elliptic curve E/¥p and an isomorphism if) : End(-E') — )■ O where is a 
maximal order in Bp^oo- Note that by using ip we can view elements of End(i?) ®Q as elements 
of Bp^oo- Given an element (p G Hom(£', E'), we obtain an embedding Hom(£", E) — End(-E) 
by mapping / 1— )■ / o 0. Thus we can view Hom(i?', E) as an ideal of End{E) or, by using the 
isomorphism ijj, as a. left ideal I of O. Note that if we replace by a different endomorphism 
0', this corresponds to right multiplying / by G Bp^oo (where 4'~^4'' = (0^0') / deg(0) 

can be viewed as an element of End(-E') CSQ). Thus a different choice of (p results in a different 
ideal, but one in the same ideal class. In fact, the map 

{supersingular curves over Fp} — )■ {left ideal classes of O} 

E' ^ ?/;(Hom(E', E)(j)) for some G Hom(E, E') 

is a bijection. Given a morphism and the ideal / = ip{B.om.{E' , E)(f)), we can also identify 
the group Hom(i?, E') with the ideal by the map 

Hom(E, E') Bp,oo, g^i' {(P~'g) . 

We would like to view End(-E") as a subring of Bp^oo in a compatible way. Let denote a 
fixed element of B.om[E,E'). Then we obtain a map 

End(E') ^ Bp,„o, g^i^ {<P''g<P) I (deg(0)) . 

Since this map is an embedding, the image O' is isomorphic to End(£") and therefore is a 
maximal order in Bp oo- One can check that O' is the right order of / := '0(Hom(i?', £')0). 
Thus, given a fixed G Hom(£', £") and a fixed isomorphism -0: End(£') — )• (9 C Bp_oo, we 
can identify End(£' x E') with the subring of M2(Bp^oo), 

O'J ' 

where / and O' are defined as above. Under this identification the Rosati involution can be 
described as 

/ N(J)^v\ 

where A^(/) := gcd{77^ | V7 G /} is the norm of I. From this point forward, we will refer 
interchangeably to elements of End(-E' x E') and elements of 'R-(E,E',<f>) '■= 

5.2. Isomorphisms of principally polarized CM abelian varieties. Two principally 
polarized abelian varieties with complex multiplication {A, li, Ai), {A, L2, A2) are isomorphic 
either if li and L2 differ by an automorphism of Ok or if there exists a G Aut(yl) such that 
a~^Li(T = L2 and Ai = a"^ X20'. We would like to describe the set of cr G Aut(£' x E') such 
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X y 

z w 




that conjugation by a preserves the polarization. Preserving the polarization is equivalent 
to 

{a^^ijaY = a-^^^'a for all ^ e End(E x E'). 

Thus (T(T^ = ±1. Let ^ G TI(e,e',^) be the element associated to a. The previous 

condition implies that q = r = and both p and s are isomorphisms, ot p = s = and q and 
r are isomorphisms. Note that the latter possibility only occurs if E and E' are isomorphic. 
Let U(^E,E',(t>) denote the set of elements in 7l(E,E',(f>) of this form, i.e. satisfying the condition 



that {P n P ^1 =±1. 



6. Counting the number of embeddings 

Let Ok denote the ring of integers of K. If the real quadratic subfield F of K has 
class number 1, then there exists an 77 G Ok such that Ok = OF[f]]- Let D denote the 

discriminant of F and let u := ^ (^D + VD^, a generator of Of- Denote Tr(?7) = ao + aiu 

and N(?7) = (3o + (3iUJ. 

Proposition 6.1. Let E, E' he supersingular curves over Fp. Fix an isomorphism ip : End{E) — )■ 
O where O is a maximal order and an element G Hom(£', E'). Determining an embedding 

l:Ok^ End(E x E') 

is equivalent to giving two elements Ai, A2 G 'TI-{e,e' ,<f>) such that 

A1A2 =A2Ai 

(*) A2 + A^ =ao + aiAi 

A2A^ =/3o + AAi. 

Proof. Let l be an embedding. Then Ai = and A2 = ^(77) satisfy the conditions above. 
In the other direction, it is clear that the map that sends a; h- )■ Ai and 77 1— )■ A2 gives a 
well-defined homomorphism from Ok to 'R-{E,E',<f>)- Therefore it remains to show that the 
map is injective. Assume that the map is not injective and let J denote the kernel. If J is 
nontrivial then Ok/ J embeds into TZ(e,e',(I>)- However, since Ok/ J is a finite abelian group 
and Bp 00 is a torsion-free Z-module, this is impossible. □ 

We note that the idea behind this lemma is similar to the ideas in |GL07j which used 
the embedding problem to obtain a bound on the primes dividing the denominator. Goren 
and Lauter studied pairs of matrices Mi, M2 in 7l{E,E',<f>) where Mi satisfies the minimal 
polynomial of \/D and M2 satisfies the minimal polynomial of a/A where K = F{y/A). 

In light of Proposition 16 . 1 1 and Section \572\ the number of pairs of {E x l), counted up 
to isomorphism, is equal to 

{ ( Ai , A2) G nf^^E',^) satisfying Sj 
E # (A,A,)^(^A,^^i,^A,^^i) -here GHom(E,E) and f/G^.,.,,) 

unordered pairs 
E,E' 
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We use this formula to count the number of embeddings. Our algorithm is sketched below 
and our implementation in Magma can be found in Appendix \M We make no claims on the 
running time of the algorithm or the implementation; the algorithm was only implemented 
to obtain the numerical evidence in Table 17.11 

6.1. Algorithm for counting solutions to the embedding problem. 

(1) (Initiahze variables) Input p, D, ao, ai, /3o, f3i, D. 

(2) (Create representation of Bp^oo) Let Bp^oo = (a, —p) where a is the maximal negative 
integer such that Bp^oo is ramified only at p and oo. 

(3) (Create representations of 'R.(e,e',4)) for each unordered pair of elliptic curves {E, E')) 
We represent each 7l(E,E',(t>) by a triple {Bi, B2, B^) where Bi is a basis for O, B2 is 
a basis for / and B^ is a basis for O'. For each isomorphism class of supersingular 
elliptic curves, let O he a maximal order isomorphic to the endomorphism ring. Then 
for each left ideal class of O let / be an integral representative and let O' be the right 
order. Take bases to give representatives of 7l[E,E',<f>) for each unordered pair {E, E'). 

(a) (Find possibilities for Ai) The condition on Ai implies that it is of the form 

( Si,i Si,2 A 

\sX^jN{I) D-s^J^ 

where Si^i is an integer in [{D — \/D)/2, {D + \^)/2] and si,2 is an element in 
/ with norm equal to 

-sl, + Dsi,i-{D^-D)/4. 

There are only finitely many possibilities for each of these, so we iterate over all 
of them. 

(i) (Find possibilities for A2) Denote the entries of A2 by tij. The condition 
on A2 + A2 implies that ^2,1 is determined by ti 2 and Ai and that ti^i and 
i:2,2 have fixed trace. The condition on A2A2 implies that ti,2; ^2,2 have 
bounded norm. This gives only finitely many possibilities for ^1,1,^1,2,^2,2 
and we iterate through them all, checking whether the pair (Ai, A2) satisfies 
all of the conditions in (*). If (*) is satisfied, then keep (Ai, A2) in the list, 
otherwise discard the pair. 

(ii) (Identify elements that differ by conjugation or automorphisms of Ok) 
There are only finitely many elements in Ui^e,e',<I))- For a fixed pair (Ai, A2), 
iterate through all elements U in U(^e,e',4)) and all a G Ant^Ox) and remove 
all pairs {a{UAiU-^),a{UA2U-^)). ' 

(4) (Count the embeddings) Count the number of pairs (Ai,A2). 

7. Numerical Data 

In Table [7!T] we examine the 13 fields studied by van Wamelen |vW99] . These 13 fields 
are the only fields K such that there exists a genus 2 curve C over Q with CM by K, and 
they are all quartic Galois cyclic extensions of Q. For each field K, we list 

(1) the factorization of the rational sixth root of the denominator of ^(r), where r is a 
CM-point of K, 

13 



(2) the conjectural value of 2(j)*Gi.CAi{K) as predicted by the Bruinier-Yang formula 
(this was computed via a Magma program), and 

(3) the number of solutions to the embedding problem, computed using Algorithm 16.11 
and the Magma code listed in Appendix \M 

The three values listed above should agree if the number of solutions to the embedding 
problem is counted with multiplicity. Unfortunately, our representation of embeddings does 
not count multiplicity. However, Yang's proof suggests that the value (ordpt+ 1) /2 is the 
multiplicity of an embedding, under certain conditions. We write the Bruinier-Yang values 
as iterated exponents, where the outer exponent denotes (ordp t + 1) /2. If there is no outer 
exponent, then this value is 1. This is the case in most examples, so it appears that most 
of the embeddings have no multiplicity. We investigate the cases where the Bruinier-Yang 
conjecture does not agree with the numerical data for the other columns in Section [7?T1 

Remark 7.1. Since all of the fields in |vW99] are Galois, there is only one CM-type up to 
isomorphism. Therefore we multiply the Bruinier-Yang formula by 1/2 since the CM-cycle 
in |BY06j is CM(K,^) + CM{K,^') where $ is not the complex conjugate of ^' . 



Remark 7.2. In the data we present in Table 7.1, we set D = 'Nf/qCDisck/f) o-s in the 
original conjecture |BY06] . In his proofs, Yang sets D = AA' which is equal to the norm of 
the relative discriminant of K/F under his assumptions, but which is not equal in general. 

7.1. Analysis of data. The first remark to make is that, in every example in the table, 
the powers of the primes in the denominators of the Igusa invariants match the number of 
solutions to the embedding problem, except for the power of 2 in four examples and the 
power of 3 in the last row. The power of 3 in the last row appears to be accounted for by 
multiplicity. The power of 2 is different in the four rows 2, 4, 9, and 11, and we explain why 
that is accounted for by cancellation in more detail in Section 17.21 

The rest of this section is devoted to explaining the discrepancies when comparing the 
Bruinier-Yang conjecture with the denominators of the Igusa class polynomials. Note that 
the six rows which are marked with a (*) are cases where the Bruinier-Yang conjecture 
is already proved by Yang, since in those cases both D and D are primes congruent to 1 
(mod 4). The two rows with a (**) in the table are not covered by the conjecture, since 
in those cases the real quadratic subfield has discriminant which is divisible by a power of 
2. Next we remark that in all the cases which are covered by the Bruinier-Yang conjecture, 
that is, where D = 1 (mod 4) is prime, the numerical data confirms the conjecture and 
shows that the 2 quantities computed for each field match, except for the power of 2 in the 
examples already mentioned above and the power of 23 in row 11. We cannot explain the 
discrepancy in the power of 23 in row 11. 
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D 


CM field 


denominators of Igusa 
invariants 


Bruinier-Yang formula 
(written multiplicatively) 


Number of embeddings 
(written multiplicatively) 


5 


Q(C5)* 


1 


1 


1 


2^ 


Q 


(V-2 + V2)" 




1 


(24)-l/2(28)-3/2 


2 


13 


q( 


\/-13 + 2yT3 


* 

) 


1 


1 


1 


2^5 




» (\/-5 + Vsj 




1, 112 


(22)3/2112 


22 - 112 


5-132 


Q( 


V-65 + 26v^ 


) 


112, 312412 


112312412 


112312412 


29 


q( 


V-29 + 2^29^ 


1 


52 


52 


52 


5-172 


Q( 


V-85 + 34v^ 


) 


712, Il24l2gi2 


II24I26I27I2 


II24126I2712 


37 


Q (^V-37 + 6V37^ 


I 


32112 


32112 


32112 


2^52 


Q (^V-10 + 5V2j 




72232, 72172232 


(228)-3/2(22)-l(2l2)-l/2 

X (532)1/251476 
X 17^23^312412792 


227^172234 


5213 


0(^-65 + 10v^ 


) 


32, 32532 


34532 


34532 


2^13 


Q( 


V-13 + 3v^ 


) 


32232, 322321312 


(22)13^2321312 


223^23^1312 


53 


Q fV-53 + 2^53 


* 

I 


172292 


172292 


172292 


61 


q( 


\/-61 + 6^61 


* 

I 


3^52412 


(32)^52412 


3252412 


Table 1. 


Numerical data: precise descriptions oi 


■ the values computed can be found in Section [7 



* : Conjecture has been proved in this case since D = 1 (mod 4) is a prime 

** : Conjecture (as stated) does not include this case; |BY06j assumed that the real quadratic subfield has 
prime discriminant 



Other than the power of 23, the discrepancies in the table occur in the four fields where 
D is highly divisible by 2. In those cases, negative and/or fractional powers of 2 appear in 
the Bruinier-Yang formula, plus other stray primes and powers in row nine. To explain that, 
observe that in [YanatlYanb] . Yang shows that the multiplicity is equal to 

and proves under his assumptions that this value is equal to 

if , n + ui^/d \ 

- ord„ hi. 

2y^2D J 

In particular, ordp is odd. However, these statements no longer hold for those 4 

examples in the table, and this leads to the fractional exponents. 

When D is also even in addition to D being even, Bruinier and Yang did not conjecture 
a formula, so the different values do not give a counterexample to the conjecture. However, 
in the aim of obtaining a formula for all CM-fields, we suggest a possible explanation for 
the differences and a possible correction. In Yang's papers, he proves that one should con- 
sider all pairs m, n such that is a positive integer divisible by p. This proof relies 
on the fact that D = 1 (mod 4). Note that if one imposes the additional condition that 
8m + n = (mod 16), then one obtains intersection numbers 1 and 7^17^23^ for the fields 

Q ^\/— 2 + and Q (^^/—10~+~5^/2^ respectively. Away from 2, these numbers agree 

with both the number of embeddings and the denominators. This suggests that when the 
real quadratic subfield has discriminant congruent to modulo 4, additional congruence 
conditions on the pairs m, n are needed to correct the intersection formula. 

7.2. Powers of 2. In four examples in the table, rows 2, 4, 9, and 11, the power of 2 in 
the denominator does not match the number of embeddings at the prime 2. The reason for 
that is cancellation of powers of 2 in the numerator and denominator of Igusa invariants. It 
was noted in jGL07t §6.2] that the phenomenon of cancellation could occur, especially for 
the prime 2. It was explained there that this is due to the fact that there are no smooth 
superspecial genus 2 curves in characteristic 2, despite the fact that superspecial primes are 
prevalent. For these four fields, the prime 2 is ramified, and so it follows from a generalization 
of the work of Goren |Gor97j given in |GLt Section 3] that 2 is a superspecial prime, and 
thus these curves have bad reduction at 2. So indeed we do find solutions to the embedding 
problem at 2 in those cases, and the only reason that 2 does not appear in the denominator 
is because of cancellation with powers of 2 in the numerator. 
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Appendix A. Magma code for counting the number of embeddings 

//Given a list of 4 rational numbers, ql, q2, q3, q4 
//return the minimal positive integer N such that 
//Nql, Nq2 Nq3, Nq4 are integers 
function Denom(L) 

denominators := [] ; 

for i := 1 to 4 do //hardcoded the length 

Append ( "denominators , Denominator (L [i] ) ) ; 
end for; 

return LCM (denominators) ; 
end function; 

//Check whether the element x is in the integral span of elements 
//in the basis B 
function InldeaKx, B) 

A := Matrix ( [Coordinates (B[l] ) , Coordinates (B [2] ) , 

Coordinates (B [3] ) , Coordinates (B [4] ) ] ) ; 

V := Vector (Coordinates (x) ) ; 

s := Solution(A,v) ; 

retval : = true ; 
for n := 1 to 4 do 

if not Islntegral (s [n] ) then 
retval := false; 

end if ; 
end for; 
return retval; 

end function; 

//Return the Rosati involution of M. 

//Notice this depends on the Norm of the ideal I 

function Rosati(M,NI) 

return Matrix([ [ Conjugate (M [1] [1] ) , Conjugate (M [2] [1] )*NI] , 

[ Conjugate(M[l] [2])/NI, Conjugate(M[2] [2] )]] ) ; 

end function; 
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//Find all elements in the integral span of the basis B with 

//norm = Normx and trace = Tracex 

function FixedNormAndTrace (Normx, Tracex, B) 

xvals := [] ; 
Q := Parent (B[l] ) ; 
isq := -Norm(Q . 1) ; 
jsq := -Norm(Q . 2) ; 

A := Transpose ( Matrix ([ Coordinates (B [1] ) , Coordinates (B [2] ) , 

Coordinates (B [3] ) , Coordinates (B [4] ) ] ) ) ; 

denoml := Denom(A[2]); 
denom2 := Denom(A[3]); 
denomS := Denom(A[4]); 
xO := Tracex/2; 
temp3 := Normx - x0"2; 

if tempS ge then 

for numS := Ceiling(-Sqrt(Rationals() ! (temp3*denom3~2/(isq*jsq)))) 
to Floor (Sqrt(temp3*denom3"2/(isq*jsq))) do 

temp2 := temp3 - (num3/denom3) "2*isq*j sq; 
for num2 := Ceiling(-Sqrt(temp2*denom2~2/(-jsq))) 
to Floor (Sqrt(temp2*denom2~2/ (-jsq))) do 

tempi := temp2 - (num2/denom2) ~2*(-jsq) ; 
for numl := Ceiling(-Sqrt (templ*denoml"2/(-isq) ) ) 
to Floor (Sqrt (templ*denoml"2/(-isq) ) ) do 

X := xO + numl/denoml*Q . 1 + 

num2/ denom2*Q . 2 + num3/denom3*Q . 3 ; 
if (Norm(x) eq Normx) and (Trace (x) eq Tracex) 
and InldeaKx, B) then 
Append ( "xvals , x) ; 
end if ; 
end for; 
end for; 
end for; 
end if ; 

return xvals ; 
end function; 
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function RemoveConjElts (solns , E, alphaO, alphal, betaO, betal , D) 



n := 1; 

while n le #solns do 



// Loop over automorphisms of E 

for r in FixedNormAndTrace(l , 0, E[l]) cat 

FixedNormAndTraceCl , 1, E[l]) cat 

FixedNormAndTraceCl, -1, E[l]) cat 

FixedNormAndTraceCl, 2, E[l]) cat 

FixedNormAndTraceCl, -2, E[l]) do 

// Loop over automorphisms of E' 
for s in FixedNormAndTraceCl, 0, E[3]) cat 
FixedNormAndTraceCl, 1, E[3]) cat 

FixedNormAndTraceCl, -1, E[3]) cat 
FixedNormAndTraceCl, 2, E[3]) cat 

FixedNormAndTraceCl, -2, E[3]) do 



//Calculating NormI so that we can compute Rosati involution 
I := LeftldealCQuaternionOrderCE[l] ) , E[2]); 
NormI : = Norm CD ; 

U := MatrixC[[r,0] , [0,s]]); 

Uinv := Matrix C [[Conjugate Cr) , 0] , [0, Conjugate Cs)]] ) ; 

//Remove embedding which is conjugate to given embedding by U 
// First check that U does not fix embedding 
if not Csolns [n] [1] *U eq U*solns [n] [1] and 

solns [n] [2] *U eq U*solns [n] [2] ) then 



Exclude C~solns , [U*solns[n] [l]*Uinv, 
U*solns [n] [2] *Uinv] ) ; 



end if ; 



//Remove embedding which is conjugate to the complex conjugate 
//of given embedding by U 

// First check that U does not fix complex conjugate of 
// embedding 

if not Csolns [n] [1] *U eq U*solns [n] [1] and 

solns [n] [2] *U eq U*Rosati Csolns [n] [2] , NormI)) then 



size := #solns; 
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Exclude ("solns , [U*solns[n] [l]*Uinv, 

U*Rosati (solns [n] [2] , NormI) *Uinv] ) ; 

end if ; 



//if E and E' are isomorphic then remove embedding which are 
// conjugate and have the order of E, E' switched 
if E[l] eq E[2] then 

U := Matrix([[0, r] , [s, 0]]) ; 

Uinv := MatrixC [[0, Conjugate (s) ] , [Conjugate (r) , 0]]); 

if U*Uinv ne Matrix ([[1,0] , [0,1]]) or 
Uinv*U ne Matrix ([ [1 ,0] , [0, 1] ] ) then 
print "Error - Uinv incorrect!!", r, s; 

end if ; 

if not (solns [n] [1] *U eq U*solns [n] [1] and 

solns [n] [2] *U eq U*solns [n] [2] ) then 

size := #solns; 

Exclude ("solns, [U*solns [n] [l]*Uinv, 

U*solns [n] [2] *Uinv] ) ; 

end if ; 

if not (solns [n] [1] *U eq U*solns [n] [1] and 

solns [n] [2] *U eq U*Rosati(solns [n] [2] , Norml)) then 
size := #solns; 

Exclude ("solns, [U*solns[n] [l]*Uinv, 

U*Rosati (solns [n] [2] , Norml) *Uinv] ) ; 

end if ; 
end if ; 
end for; 
end for; 

n := n + 1; 

end while; 

return solns; 

end function; 



// find Lambdal, Lambda2 satsifying embedding conditions and such that 
// entries have fixed norm 

function FindSolns(E, alphaO, alphal, betaO, betal, D, 

Normsl2, Normtl2, Normt22, Tracet22, 
Normtll, Tracetll, sll) ; 

solns := [] ; 



Rl := QuaternionOrder(E[l] ) ; 
R2 := Quaternion0rder(E[3] ) ; 
I := LeftldeaKRl, E[2]); 

// Create list of possible values in I with Norm = Normsl2 
sl2traces := [-Floor (Sqrt (4*Normsl2) ).. Floor (Sqrt (4*Normsl2) )] ; 
sl2vals := [] ; 
for T in sl2traces do 

sl2vals := sl2vals cat FixedNormAndTrace(Normsl2, T, E[2]); 
end for; 

for sl2 in sl2vals do 

// loop over values in Rl with Norm = Normtll and Trace = Tracetll 
for til in FixedNormAndTrace (Normtll , Tracetll, E[l]) do 

// loop over values in R2 with Norm = Normt22 and Trace = Tracet22 
for t22 in FixedNormAndTrace (Normt22, Tracet22, E[3]) do 

// Create list of possible values in 1 with Norm = Normtl2 
tl2traces := [-Floor(Sqrt(4*Normtl2)) . .Floor(Sqrt(4*Normtl2))] ; 
tl2vals := [] ; 
for T in tl2traces do 

tl2vals := tl2vals cat FixedNormAndTrace (Normt 12, T, E[2]); 
end for; 

for tl2 in tl2vals do 

t21 := l/Norm(I)*(alphal*Conjugate(sl2) - Conjugate (tl2) ) ; 

//define Lambdal = omega and Lambda2 = eta 
omega := Matrix( [ [sll , sl2] , 

[Conjugate (sl2) /Norm(l) , D - sll]]); 
eta := Matrix( [ [til , tl2] , [t21, t22]]); 
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//check omega and eta live in correct subring 
// and satisfy all of the embedding conditions 
if Inldeal(sl2, E[2]) and Inldeal(tl2, E[2]) and 
(omega*eta eq eta*omega) and 

eta + Rosati(eta, Norm(I)) eq alphaO + alphal*omega and 
eta*Rosati (eta, Norm(I)) eq betaO + betal*omega then 

Append ( ~solns , [omega, eta]); //add to list of solutions 
end if ; 
end for; 
end for; 
end for; 
end for; 
return solns; 

end function; 



// Lists all possible pairs of matrices Lambdal, Lambda2 in R_{E, E'} 
// such that 

// Lambdal *Lambda2 = Lambda2*Lambdal 

// Lambdal "2 - D*Lambdal + 1/4* (D"2 - D) = 

// Lambda2 + Lainbda2~{\vee} = alphaO + alphal*Lambdal 

// Lambda2*Lambda2~{\vee} = betaO + betal*Lambdal 

function List AllEmb (alphaO, alphal , betaO, betal , D, p) 

// Define constants associated to CM field 
F := QuadraticField(D) ; 
D := Discriminant (F) ; 

trw := D; //Trace (omega) 

nw := Integers ! (1/4* (D"2 - D)); //Norm(omega) 

Dtilde := l/16*D~4*alphal~4 + l/2*D"3*alphaO*alphal"3 - l/8*D"3*alphal~4 

- D"3*alphal"2*betal + 3/2*D"2*alphaO"2*alphal"2 

- l/2*D"2*alphaO*alphal~3 - 4*D"2*alphaO*alphal*betal 

+ l/16*D"2*alphal"4 - 2*D"2*alphal"2*betaO + D"2*alphal~2*betal 
+ 4*D"2*betal"2 + 2*D*alphaO~3*alphal - l/2*D*alphaO~2*alphal"2 

- 4*D*alphaO~2*betal - 8*D*alphaO*alphal*betaO 

+ 4*D*alphaO*alphal*betal - 2*D*alphal"2*betaO + 16*D*betaO*betal 

- 4*D*betal"2 + alphaO"4 - 8*alpha0"2*beta0 + 16*beta0"2; 
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// Give a presentation of B_{p, infinity} 
if p ne 2 then 

jsq := -p; 

isq := -1; 

while (IsSquare(FiniteField(p) ! isq)) or 

(#Ramif iedPrimes (QuaternionAlgebra<Rationals I isq, jsq>) 

gt 1) do 

isq := isq - 1; 
end while; 

else 

jsq := -1; 
isq := -1; 
end if ; 

QQ := RationalsO; 

Q<i, j, k> := QuaternionAlgebra<QQ I isq, jsq>; 
ksq := -isq*jsq; 

// Orders ={0:0 maximal order isomorphic to End(E) for E supersingular 
// curve over Fpbar} 

// Note: if E and E' are not isomorphic, but have isomorphic endomorphism 

// rings. Orders contains two different maximal orders, one is identified 

// with End(E) , the other with End(E') 

00 := MaximalOrder(Q) ; 

LI := LeftldealClasses(OO) ; 

Orders := [] ; 

for I in LI do 

Append ( "Orders , RightOrder (1) ) ; 
end for; 

// Ends = list of triples, one triple for each (ordered) pair (E, E') 

// Let be a maximal order which is isomorphic to End(E) , I a left ideal 

// of which can be identified with Hom(E' , E) , and 0' the right order of 1 

// which is identified with End(E'). Then the triple associated to (E, E') 

//is (Basis (0), Basis (1), Basis(OO). The correspondence between 

// (0, I, 0') and (E, E') is explained more in the paper. 

Ends := [] ; 

for in Orders do 

for 1 in Lef tldealClasses(O) do 

Append(~Ends, [Basis(O), Basis(I), Basis(RightOrder(I))] ) ; 

end for; 
end for; 

// initialize list of solutions 
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solns := [] ; 
number := 0; 

//look for Lambdal, Lambda2 in R_{(E, E')} satisfying above conditions 

// this is equivalent to searching for sll, til in 0, sl2, tl2 in I, 

// s22, t22 in 0' . Then 

// Lambdal = 

// [sll sl2] 

// [sl2~{\vee} s22] 

// Lambda2 = 

// [til tl2] 

// [tl2~{\vee} t22] 

for E in Ends do 

NormI := Norm(LeftIdeal(QuaternionOrder(E[l] ) , E[2])); 

//possible values for sll. s22 = D - sll 

sllvals := [CeilingC (D - Sqrt(D))/2 )..Floor( (D + Sqrt(D) )/2)] ; 

Esolns := [] ; 

for sll in sllvals do 

a := sll; //remnants of old notation 
delta := -a~2 + a*trw - nw; 
m := delta; 

Tracetll := alphaO + alphal*sll; //conditions from Lambda + Lambda* 
Tracet22 := alphaO + alphal*(D - sll); 

for n := Ceiling(-Sqrt (Max(m"2*Dtilde - 4*D, 0))) to 
Floor (Sqrt(Max(m"2*Dtilde - 4*D, 0))) do 

//parts of old notation 

Normx := l/(2*D)*(n - (l/4*a"2*D"2*alphal~2 

- l/4*a*D"3*alphal"2 + l/16*D~4*alphal"2 - l/4*a"2*D*alphal"2 
+ l/4*a*D"2*alphal"2 - l/8*D"3*alphal~2 

+ a"2*D*alphal*alphaO - a*D"2*alphal*alphaO 
+ l/4*D"3*alphal*alphaO + l/16*D"2*alphal"2 

- l/4*D"2*alphal*alphaO + a"2*alpha0"2 - a*D*alphaO"2 
+ l/4*D"2*alphaO~2 - 2*a"2*D*betal + 2*a*D"2*betal 

- l/2*D"3*betal - l/4*D*alphaO"2 - 2*a*D*betal 
+ l/2*D"2*betal - 4*a"2*beta0 + 4*a*D*betaO 

- D"2*beta0 - D*betaO)); 

Normu := delta* (betaO + betal*a) - delta*Normx; 
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Tracexuc := betal*delta - (D - 2*a)/delta*Normu; 

Normv := delta~2*Normx + delta* (D - 2*a)*Tracexuc 
+ (D - 2*a)"2*Normu; 

//conditions coming from fixed n 
Normtll := Normx; 
Normtl2 := Normu*NormI/delta; 
Normt22 := Normv/delta"2 ; 
Normsl2 := NormI*delta; 

//check n can give a solution which is integral 
if IsIntegral((m"2*Dtilde - n"2)/(4*D*p) ) and 

Islntegral (Normtll) and lslntegral(Normtl2) and 

IsIntegral(Normt22) then 

Esolns := Esolns cat FindSolns(E, 

alphaO, alphal, betaO, betal, D, 
Normsl2, Normtl2, Normt22, Tracet22, 
Normtll, Tracetll, sll) ; 

// finds all solutions with 

// entries of Lambda2 having 

// fixed norm and fixed 

// trace (for some entries) 

end if ; 
end for; 
end for; 

// remove elements which are conjugate 

Esolns := RemoveConjElts (Esolns , E, alphaO, alphal, betaO, betal, D) ; 

if E[l] eq E[2] then 

number := number + #Esolns; 

else 

number := number + #Esolns/2; 
end if ; 

Append ( ~ solns , Esolns); 
end for; 

return solns, Ends, Integers () ! number; 
end function; 
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